The present invention relates to a focus control method and an image input apparatus employing the method and, more particularly, to a focus control method for sensing an image of high resolution by performing pixel shifting operation, i.e., an operation for shifting an image formation position of an incoming optical image on a solid-state image sensing device, and an image input apparatus employing the control method.
FIG. 4 is a block diagram briefly illustrating a configuration of a conventional image input apparatus, and FIGS. 5A to 5P are explanatory views showing pixel shifting operation shifting by half pixel.
In FIG. 4, reference numeral 101 denotes a lens unit including a focusing function; 102, a solid-state image sensing device (simply referred by "image sensing device", hereinafter), such as CCD, covered with a complementary color (green (Gr), yellow (Ye), magenta (Mg), cyan (Cy)) mosaic filter; 104, a sample-and-hold (S/H) circuit for sampling and holding image signals; 105, an automatic gain controller (AGC) for performing automatic gain control; 106, an analog-digital (A/D) converter for converting analog signals to digital signals; 601, an image memory for storing a complementary color image; and 602, a memory controller for controlling the image memory 601.
Further, reference numeral 108 denotes a color demodulation separator which converts input signals to primary color signals (R, G, and B signals) and outputs them; 109, a processor which performs signal processes, such as white balance processing and .gamma. correction, and outputs a luminance signal and color difference signals; 111, an encoder which generates a video signal of a predetermined format on the basis of the luminance signal and the color difference signals; 112, an interface (I/F) for outputting the video signal; 114, a focus motor driver for driving a focus motor provided in the lens unit 101; 115, an image sensing device operation unit for operating the image sensing device 102; 117, a band-pass filter (BPF) for extracting high frequency components to be used for determining a focus state from image signals; 118, an accumulator for accumulating data (evaluation value) to be used for obtaining the direction and speed at which the lens unit 101 is to be moved (referred by "focusing data", hereinafter) using the outputs from the BPF 117; 119, a focus controller for controlling the focus motor driver 114; 120, a CPU for controlling the entire operation of the image input apparatus; 121, a synchronizing signal generator for generating a synchronizing signal; 122, a timing signal generator (TG) for generating a timing pulse for operating the image sensing device operation unit 115 on the basis of the synchronizing signal from the synchronizing signal generator 121; 125, a .gamma. correction controller for controlling the .gamma. correction processing; 126, a white balance controller for controlling the white balance processing; 127, a known shift controller for performing pixel shifting which is controlled by the CPU 120; and 128, an input unit, such as a keyboard and a pointing device.
When colors of the complementary color mosaic filter are arranged as shown in FIG. 6, the CPU 120 is able to operate the timing signal generator 122 in two different modes by using a control signal (FR). One is a normal mode in which image signals are read while combining charges of two pixels adjoining in the horizontal direction, as A1, A2, B1 and B2 in FIG. 6. The other is a frame mode, and image signals are independently read by every other line, as C1, C2, D1 and D2 in FIG. 6, without combining charges of two pixels adjoining in the horizontal direction. It is assumed that the above conventional image input apparatus reads image signals in the frame mode when performing pixel shifting operation, and in the normal mode otherwise.
Next, an operation of the image input apparatus having the above configuration will be described below.
Light incoming through the lens unit 101 is converted into electric charges by the image sensing device 102. Obtained electric signals, i.e., image signals, are sampled by the S/H circuit 104, then amplified by a controlled gain in the AGC 105. The amplified image signals are converted into digital signals by the A/D converter 106 and stored in the image memory 601 as a complementary color image. The complementary color image signals stored in the image memory 601 are read under control of the memory controller 602 to the color demodulation separator 108 where the complementary image signals are converted into R, G and B signals.
Different processes are performed for converting the complementary image signals into the R, G and B signals when the image signals are read in the normal mode and when the image signals are read in the frame mode. First, a case where the image signals are read in the frame mode is explained.
The image signals are independently read while performing pixel shifting operation as shown in FIG. 5A to FIG. 5P. With the pixel shifting operation, image data of four colors, namely Ye, Cy, G and Mg, at sampling points which are four times finer than the pixel positions of the image sensing device 102 is obtained. The image data obtained as above is converted into R, G and B signals by performing the following matrix operation. ##EQU1##
Next, a case where the image signals are read in the normal mode is explained.
As seen in FIG. 6, when image signals are read while combining charges of two pixels adjoining in the vertical direction in, e.g., the first and second lines (A1), the third and fourth lines (A2), and so on, then every four lines, image signals corresponding to the sum of the charges in pixels at Mg filter positions and at Ye filter positions (Mg+Ye), the sum of the charges in pixels at G filter positions and at Cy filter positions (G+Cy), the sum of the charges in pixels at Mg filter positions and at Cy filter positions (Mg+Cy), and the sum of the charges in pixels at G filter positions and at Ye filter positions (G+Ye) are obtained. With these four kinds of image signals, a luminance signal and color difference signals are generated by using the following equations. ##EQU2##
Similarly, when image signals are read while combining charges of two pixels adjoining in the vertical direction in, e.g., the second and third lines (B1), the fourth and fifth lines (B2), and so on, four kinds of image signals, namely, (Mg+Ye), (G+Cy), (G+Ye), and (Mg+Cy), are obtained every four lines. Therefore, the luminance signal and the color difference signals are also obtained by using the equations (2).
With the luminance signal and the color difference signals obtained as above, the R, G and B signals are generated by performing the following matrix operation. ##EQU3##
The processor 109 performs .gamma. correction and white balance processing under control of the .gamma. correction controller 125 and the white balance controller 126, and outputs color difference signals, R-Y and B-Y, and a luminance signal Y. The output signals are converted into video signals by the encoder 111, then outputted to outside via the I/F 112. The synchronizing signal generator 121 generates a synchronizing signal, and the timing signal generator 122 generates a pulse signal on the basis of the synchronizing signal. The image sensing device operation unit 115 and the focus controller 119 controls the image sensing device 102 and the focus, respectively, in accordance with the pulse signal. Further, the S/H circuit 104, the processor 109, and the encoder 111 also operate in synchronization with the synchronizing signal.
In such the image input apparatus, a single image of high resolution is obtained by performing pixel shifting operation in the following manner. First, light path of an incoming optic al image or the image sensing device 102 is shifted by a predetermined amount (e.g., half pixel) by the shift controller 127 so as to interpolate image data between pixels, as shown in FIGS. 5A to 5P, and an image is taken at each shifted position to obtain a plurality of images (16 images in the case of FIGS. 5A to 5P). Thereafter, the obtained plurality of images are combined to generates a signal image of high resolution.
Further, in a case of performing an automatic focusing operation, when pixel shifting operation shifting by half pixel is performed, the high frequency components, which are necessary for determining a focus state, are extracted from image signals, outputted from the processor 109 by using RGB image data of a single image which is generated from the complementary color image data of 16 images sensed at the positions shown in FIGS. 5A to 5P, by the BPF 117. Then focusing data is accumulated by the accumulator 118 from the extracted high frequency components, and the focus controller 119 calculates the direction and speed at which the lens unit 101 is moved on the basis of the accumulated focusing data. Finally, the focus motor driver 114 drives the focus motor to move the lens unit 101.
In another way of performing the automatic focusing operation, an image is focused first without performing pixel shifting operation, then the position of the lens unit 101 is fixed when the image is focused. Thereafter, pixel shifting operation is performed to obtain an image of high resolution.
With the above configuration of the conventional image input apparatus, in order to obtain an RGB image of high resolution from a plurality of complementary color images by performing pixel shifting operation, color component data of four colors, namely three complementary colors, magenta, cyan and yellow, and green are needed for each pixel of the RGB image of high resolution. Therefore, if the pixel shifting operation shifting by half pixel is performed, as shown in FIGS. 5A to 5P, the aforesaid color component data of four colors for each pixel of the RGB image of high resolution can not be obtained until sixteen image sensing and data taking operations are completed. Thus, it takes a considerable time for the image input apparatus to generate image signals which are processed with various color processing operations. Furthermore, in a conventional image input apparatus which accumulates data for an automatic focusing operation or an automatic iris control operation by using processed image signals, there is a time gap before the iris is adjusted and an image is properly focused.
Further, in the method for obtaining an image of high resolution by focusing the image input apparatus on the basis of an image obtained before performing pixel shifting operation and fixing the position of the lens unit 101 when the image is focused, then performing the pixel shifting operation for shortening the time to focus on the image, since the resolution of the image used for focusing the image input apparatus is lower than the resolution of an image obtained after pixel shifting operation, there is a problem in which the resultant resolution of the image of high resolution is low because only a low focus level is achieved.